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ABSTRACT 

We concern ourselves with the hypothesis that two 
variables have a perfect disattenuated correlation, hence measure the 
same trait except for errors of measurement. This hypothesis is 
equivalent to saying, within the adopted model, that true scores of 
two psychological tests satisfy a linear relation. Statistical tests 
of this hypothesis are derived when the relation is specified with 
the exception of the additive constant. Two approaches are presented 
and various assumptions concerning the error parameters are used. 

Then the results are reinterpreted in terms of the possible existence 
of an unspecified linear relation between true scores of two 
psychological tests. A numerical example is appended by way of 
illustration. (Author) 
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TESTING A LINEAR RELATION BETWEEN TRUE SCORES OF WO MEASURES 



Walter Kristof 
Summary 

We concern ourselves with the hypothesis that two variables have a perfect 
disattenuated correlation, hence measure the same trait except for errors of 
measurement. This hypothesis is equivalent to saying, within the adopted model, 
that true scores of two psychological tests satisfy a linear relation. Statisti 
cal tests of this hypothesis are derived when the relation is specified with the 
exception of the additive constant. Two approaches are presented and various 
assumptions concerning the error parameters are used. Then the results are 
reinterpreted in terms of the possible existence of an unspecified linear rela- 
tion between true scores of two psychological tests. A numerical example is 
appended by way of illustration. 
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TESTING A LINEAR RELATION BETWEEN TRUE SCORES OF TWO MEASURES 1 
1. Introduction 

Let X = T 4 E and Y = U 4 F be two random variables made up of true 
scores T , U and errors of measurement E , F . Suppose there is a linear 
relation between T and U . Th< n X and Y may be viewed as measuring che 
same dimension, but each individual measurement will be disturbed by an error. 
This error will cause X and Y to be less than perfectly correlated although 
the disattenuated correlation will still be perfect. 

The statistical problem of deciding whether a disattenuated correlation 
may be assumed to be perfect is of obvious practical significance. However, 
earlier techniques proposed for this purpose (Forsyth & Feldt, 19^9; 1970; 

Lord, 1957; McNemar, 1958) have been recently assess id by Lord (1971) as 
"cumbersome, approximate, or flawed." 

Lord (1971) suggested instead a procedure based on the construction of a 
confidence interval for the coefficients of a linear relation between true 
scores. This procedure is an adaptation of a result by Villegas (1964). 

However, simple techniques for testing whether a disattenuated correlation 
is perfect can be obtained even when the statistical assumptions underlying 
Lord's suggestion are considerably relaxed. The derivation of such techniques 
is the main concern of this paper. 

In particular, we plan to proceed as follows, (i) Statistical tests will 
be derived for the hypothesis that there is a linear relation between true 
scores of two measures. This relation will be specified with the exception of 
the additive constant. Two approaches will be presented, and various assump- 
tions concerning the error parameters will be used, (ii) The results will be 

Research reported in this paper has been supported by grant GB-18250 from 
National Science Foundation. 
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reinterpreted in terms of the possible existence of an unspecified linear 
relation between true scores, (iii) A numerical example will be given by 
way of illustration. 



2. First Approach to Testing a Linear Relation Between True Scores 

Let X,Y be observed scores, T,U true scores and E,F errors of 
measurement on two psychological tests. Suppose that each test has been 
divided into two parts with observed scores X^,Xg and Y^, Yg and errors 
E^Eg and F-^Fg We will write E' = - Eg and F' = F, - Fg . Let 

the division of the tests be such that true scores on the parts of a given 
test may differ only by a constant. We introduce the following four variables? 

X =X 1 + Xg = T+ E 
X’ =X 1 -Xg = a+ E' 

(1) 

Y = Y 1 + Yg = U + F 
Y' = Y 1 - Yg = b + F' , 



a and b being constants . 

The following assumptions concerning second moments of the errors will be 

made : 

/ . \ 2 2 . 2 2 

(i) \ = % “ d \ - \ 

(ii) Variables E^,F.g,F^,Fg are pairwise independent with the 

possible exception <r_ = ct £ 0 

E 1 F 1 E 2 F 2 



(iii) Variables T,U are independent of variables E, ,Eg,F, ,Fg 
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Assumptions (i), (ii), (iii) are equivalent to the following assumptions. 

(!') o- E =<r E , and <r F = d F , 

(ii : ) Variables E,E',F,F' are pairwise independent with the possible 
exception = <r E , pt / 0 . 

(iii') Variables T,U are independent of variables E,E’,F,F' . 

This set of assumptions is somewhat weaker than the usual set of assumptions 
about error second moments in classical test theory when repeated measurements 
have been obtained. For, cr ,, = or and o-„_ = ov,,-, need not be zero. 

^ iJjgJ? g iii i? 

The linear hypothesis we wish to test on the basis of a sample of 
observations is 



( 2 ) 



H q s P X T + PgU + 7 - 0 (P 1 i 0) 



with specified coefficients p^ and Pg . The additive constant y remains 
unspecified. 

Let us introduce the following new variables: 



( 5 ) 



Z • = p.X + P 0 Y = p.T + P C U + p.E + P 0 F 



Z' = P.X 1 + p 0 Y' = p.a + p_b + p.E' + p 0 F' 



The variances become 



( 4 ) 



2 2 
^p^+PgU + ^p^PgF 



cr„, = cr, 



z' p x e+p 2 f * 



In addition, Z and Z' are independent regardless of p^,Pg 
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Hence we have 



( 5 ) 




2 

^T+^U 




Precisely when Hq is correct this reduces to 



( 6 ) 




This result enables us at once to devise the desired statistical test. 

The details of this test will depend on additional assumptions that we are 
willing to make. We will consider two cases. 

Case I : Suppose that each test has been split such that true scores on 

the parts of a given test are equal. This implies a = b = 0 . Suppose fur- 
ther that expected errors on the two parts of a given test are equal. This 

implies expectations zero for E' and F* • Hence Z' will have expectation 
zero. 

Case II : Suppose that a and/or b need not be zero. Suppose further 

that expected errors on the two parts of a given test need not be equal. This 

implies possibly nonzero expectations for E' and F' . Hence Z' need not 

have expectation zero. 

It will be assumed in each case that errors are multinormally distributed. 
However, no distributional assumptions concerning T and U need be made 
under H Q . Then, considering (6) and the independence of Z and Z' under 
H q , we obtain at once the following results when a random sample of size N 
is given. 
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Case I : Let s^, be the observed sample variance of Z about the sample 

2 

mean with df = N - 1 . Let s^, be the observed sample variance of Z' 
about zero with df = N . Then, under Hq , the ratio 

m ,4 

S Z’ 

follows an F distribution with df^ = N - 1 and dfg = N . This provides 
the desired test. 

It will be convenient to express (7) in terms of the original observed 

second moments. Let V be the observed sample variance -covariance matrix of 

s 

total scores X = X^ + Xg and Y = Y^ + Yg (taken about means with df = N - 1 ). 
Let V*' be the observed sample variance -covariance matrix of difference scores 
X' = - Xg and Y' = Y^ - Yg (taken about zero with df = N ). Introduce 

the row vector j3' = ((3^,Pg) . Then (7) becomes 

Hq will be accepted if this ratio does not exceed the critical value of 

the F distribution with df^ = N - 1 , dfg = N when the chosen level of 
significance is a . Otherwise Hq will be rejected. We may also say that 
acceptance of Hq is implied precisely when 

(9) , g'(V s - ^>£<0 . 

2 

Case II: The previous definitions of s„ and V_ are retained. However, 

L *“S 

2 

let s_, be the observed sample variance of Z' about the sample mean with 

u 

df = N - 1 . Then, under H Q , the ratio 
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( 10 ) 




follows an F distribution with df^ = dfg = N - 1 . This provides the 
desired test. 

Let V, be the observed sample variance -covariance matrix of difference 
— d 

scores X' = X 1 - X g and Y* = \ ~ (taken about means with df = N - 1 ). 
Then (10) becomes 



vector as defined before. Hq will be accepted if this ratio does not exceed 
the critical value F^, of the F distribution with df^ = dfg = N - 1 when the 
chosen level of significance is 0! . Otherwise Hq will be rejected. We may 
also say that acceptance of Hq is implied precisely when 



Case II is obviously more general than case I. Any set of data satisfying 
the assumptions of case I may be subjected to the test developed for case II. 
The corresponding loss of power will be quite negligible unless N is very 
small. 

Remarks : It has been assumed that each of two psychological tests can be 

split into two parts. It will be recognized that the statistical developments 
apply as well if two forms of each test are available. The assumptions of case 
I are met if two corresponding forms are parallel. We are dealing with case II 
if the equality of means requirement is dropped. Equality of variances of two 
corresponding- forms is still postulated, however. 



(n) 



F = g'Y! * 






( 12 ) 
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It is seen that the previous (and following) developments can be general- 
ized. Tests of one linear relation between true scores of more than two 
psychological tests are obtained if the pattern of the previous derivations 
is followed. However, the existence of one linear relation alone would then 
not imply that each two tests have a perfect disattenuated correlation. In 
fact, true scores of p tests must obey p - 1 independent linear relations 
in order that this property may hold. 

The testing procedure suggested by Lord (1971 ) and adapted from a result 
by Villegas ( 196 ^) presupposes that true scores on the two parts of a given 
test are equal and that errors on the two parts have equal variances and, 
expressly, expectations zero. These assumptions are stronger than those of 
our case I. We might speak of a case 0 if we were to include the Lord/villegas 
approach into our classification of cases. 

3 . Second Approach to Testing a Linear Relation Between True Scores 

The approach presented in this section is a correlational one. It may be 
somewhat less direct but it will allow us to dispense in a very natural way 
with the previous equality of variances requirement of errors on the two parts 
of a psychological test (or the two equivalent forms). We will again limit 
ours elves to two psychological tests; the extension to p > 2 such tests is 
straightforward . 

Observed scores on the two parts of a psychological test will again be 

signified by X-^Xg and Y^Yg and the associated errors of measurement by 

E^,Eg and F^,Fg . True scores on the parts will be written as T^,Tg and 

U^,Ug with sums T = T^ + Tg and U = + Ug • Differences T^ - Tg and 

U - Up are supposed to be constants. The previous assumptions concerning 
1 2 
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second moments of errors along with their normal distribution are retained. 
The hypothesis Hq to be tested is again (2). 

Let us now introduce the following new variables: 



W = = P^ + p 2 u x + p 1 E 1 + p 2 F 1 

(15) 

w = P 1 X S + p 2 Y 2 = Pl T 2 + p 2 u 2 + P X E 2 + p 2 F 2 . 

Under H the sums p^ + Pg^ and PgTg + pgUg are constants. Variables 
W and W are independent precisely when H„ is true. However, if Hq is 
false, then W and W* will be positively correlated since 



(i'O 



1 2 

°WW' " <J P 1 T i +P 2 U 1 ^1 T 2 +P 2 U 2 * (J V 1 ^2 l! 



Testing Hq will amount to testing whether a population correlation coeffi- 
cient may be as sumed to be zero. This is the basic result of the present 
approach. 

We are in a position to rederive the previous testing techniques . This 
is not surprising at al 3 because the previous variables (3) and the present 
variables (13) are linear transforms of each other: W = (Z + Z' )/2 and 

W' = (Z - Z' )/2 . 

Case I : Under H Q variables W and W' have equal expectations and 
equal variances. Hence the statistical problem consists in testing whether 
the intraclass correlation between W and W' may be assumed to be zero. 

A test of this hypothesis can be written down at once when we follow 
Scheffe' (1959, pp. 223-227). His SS A is (N - l)/2 times the observed 
sample variance of W + W' taken about the sample mean with df = N - 1. 

T aki ng (13) into account we derive SS A = £*'V S £(N - l)/2 . Similarly, Scheffe' s 
SS 0 becomes P'VggN/2 . This yields exactly the F test as given in (8). 
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Case II : Under Hq variables W and W' have equal variances but not 

necessarily equal expectations. Hence we have to test whether W and W' may 

2 2 

be uncorrelated when we know that cr TT = cr TT1 . 

W W' 

To this end we introduce the quantity 



(15) 



u = 



wW' 



ww 



+ S W'W' 



where s w > s ww' 



and s TTtTTt denote the usual observed sample second moments 
WW 



of W and W' taken about sample means ( N - 1 in denominators). Quantity 
u is the maximum-likelihood estimator of the correlation between W and W' 



2 2 

when these variables are binormally distributed and cr^ = cr^, 



Under Hq the 



ratio 



(16) 



F = 



1 + u 
1 - u 



follows an F distribution with df^ = df^ = N - 1 (Kristof, in press). 
Taking (13) into account it is seen that (l6) is identical to the F test (ll). 
One-sidedness of this F test reflects the fact that covariance (l4) is non- 



negative when Hq is false. 



The present correlational approach enables us at once to drop the earlier 
2 2 2 2 

assumptions or = cr_ , or = cr and ou _ = a . _ . This new situation 

E 1 E 2 F 1 F 2 E 1 F 1 E 2 F 2 

will be named 



Case III : Under Hq variables W and W' need not have equal expecta- 
tions nor equal variances . The problem reduces to a very familiar one : We 

have to test whether W and W' may be uncorrelated when no restrictions 
are imposed on means and variances . 

Obviously Hq will be retained if the sample correlation coefficient 
r^, does not exceed a certain upper limit. Such limits corresponding to 
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various levels of significance are easily calculated from the t distribution 
•with df = N - 2 . It should be borne in mind that (l4) requires us to employ 
a one-sided statistical test. 

Let the sample variance -covariance matrix V of the partitioned vector 
(X^yJ Xg.Yg) be partitioned accordingly: 




and Vgg positive definite. We may now write 

£'Vtp£ 

(18) r, = 22 : . 

Knowledge of this quantity enables us to carry out the test of . 

Vj 2 cannot be expected to be symmetric. However, in (l8 ) we may replace 
ll2 by the symmetric matrix (V 1 p + )/2 without altering the numerical 

value of r w , . In fact, 2^'V^ = £' (V^ + 

4. Testing an Unspecified Linear Relation Between True Scores 

The previous results enable us to test hypotheses about vector £ when 
the existence of a linear relation between true scores of two psychological 
tests is assumed. Depending upon additional assumptions, formulas (8), (11 ) 
or (l8) may be used. Conversely, these formulas can be utilized in constructing 
"confidence intervals" for P . It is obvious that p can be replaced by any 
nonzero multiple of it. 

However, a linear relation between true scores may not exist at all. This 
is equivalent to saying that the dis attenuated correlation between two 



12 



- 11 - 



psychological tests is less than perfect. In fact, we may be primarily 
interested in testing whether there is any linear relation between true scores. 

The assumption of such an unspecified linear relation will be denoted by Hq . 

It is our aim now to give statistical tests of Hq . 

Cases I and II : If Hq is correct, then there is (up to multiplication) 

precisely one nontrivial linear relation between true scores, (3^T + p^U +7 = 0 , 
say, with unknown values (3^,Pg 7 • There cannot be another independent 

linear relation if T and U are to be variables and not constants. Hence, 
in order to test Hq , we would use (8) or (ll), depending on the case, if £ 
were known. Now, since is unknown, let us find the quantity 



(19) 



. £%£ 

F = min ZT7S5 



in case I and 

( 20 ) 




in case II when the min imum is taken over all vectors £ ^ 0 . Let us agree 
to retain Hq exactly when F does not exceed the upper critical value F^ 
of the F distribution with, df^ = N - 1 , dfg = N in case I and df^ = dfg = 
N - 1 in case II. This will produce a conservative test of Hq . If rejection 
of Hq occurs, then the true corresponding level a will not exceed ct , 
a < a . 

Calculation of F is not difficult. F is the smallest eigenvalue of 

V in case I and of V V,^ in case II. 

— s—d — s— a 

Case III : Considerations similar to those in cases I and II lead us to 

seek the minimum of r^, when £ varies over all nonzero vectors. This 

.13 

O 
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minimum may be compared with the critical value (level a , df = N - 2 ) of a 
product -moment correlation under normality assumptions when the population cor- 
relation is zero. We have to use a one-sided test; only a sufficiently large 
positive value of the minimal r^, will lead to rejection of Hq . Again, 
the resulting test will be conservative. 

According to (l8) the minimum of r™, may be written as 

ww 



( 21 ) 



i 

min ,., 1 — i — 

yo yaiuffi' Vg a & 



We will think of as a symmetric matrix; it has already been remarked that 

substitution of (V^ + V^o)/2 for is allowable. We will further suppose 

that V-jp is positive definite. Otherwise r would not be a positive quantity 

and rejection of would not be possible at any level 0! . 

_ 2 

In order to determine r we may proceed as follows. Let = PAP * 

with P orthogonal and A positive diagonal and define 



( 22 ) 



We derive that 
(23) 



i = > 111 = i 

A = A'Vv^'^ = ||a id H 

B = = Hb^H . 



(?) -2 = max £'A^'B£ . 

UI=1 



ETS. 



2 

This procedure was developed in cooperation with Bary G. Wingersky of 
A program is available from him. 
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Writing ^ = (cos 4> , sin 4> ) we seek maximization of the function t|( 4>) = 
under variation of <t> . The condition dT)/d<t> = 0 becomes 

( 2 *0 g^cos^4> + g^cos"^ sin 4> + ggcos 2 ^ sin 2 4> + g^cos <t> sin^4> + g^sin^ = 0 
with coefficients 

S 4 = a ll b l2 + a l2 b ll 

g 3 = a n^ b 22 " b n^ + b n^ a 22 ■ a n^ + 4a i2 b i2 

( 2 5) g 2 = 3[a- 1 p(bpp - b^) + b 22^ a 22 " a ll^ 

g l = a 22^ b 22 b ll^ + b 22^ a 22 “ a U^ " ^ a 12 b l2 
g 0 = " a l2 b 22 " a 22 h l2 

In (24) 4> is no longer a variable; however, it is convenient to retain the 

same symbol. 

We see that sin <t> = 0 is a solution of (24) precisely when g^ = 0 . 

It suffices to solve the quartic 

( 26 ) g(<>) = g^ctn^4> + gyrhA + g 2 ctn 2 <t> + g^tn ♦ + g Q = 0 

which reduces to a cubic precisely when g^ = 0 . Hence there will always be 
four solutions of which at least two must be real because ti( 4>) certainly 
assumes a maximum and a minimum. 

5* Application 

Lord (1957) reported the following observed variance -covariance matrix 
for N = 649 ! 



o 
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/ 86.3979 


57.7751 


56.8651 


58.8986) 


57.7751 


86.2632 


59.5177 


59 . 6683 


56.8651 


59.5177 


97.2850 


75.8201 


y 58.8986 


59.6683 


75.8201 


97.8192^ 



Hie first two variables represent parallel halves of a vocabulary test 
administered under very liberal time limits. The last two variables represent 
parallel halves of a vocabulary test constructed so as to be as nearly equiva- 
lent as possible to the halves of the first test except that the time of 
administration was so short that only two per cent of the examinees completed 
each half. 

We wish to test the hypothesis that the two tests measure the same trait 
except for errors of measurement regardless of speed conditions. 

It appears that the data should satisfy the assumptions of case I. How- 
ever, matrix cannot be reconstructed from the available data. We will 

therefore proceed according to case II and, for reasons of comparison, also 
according to case III. We expect that the two methods should yield nearly 
identical results because of the parallelism of corresponding test halves. 

Case II: From the given data we find V and V. to be 

— s — d 

v = / 288.2113 234.74-97 \ ^ y = / 57 . 1109 -1.6829 \ 

~ s V 234.7497 34 - 2 . 74 - 4 - 4 / , \ -1.6829 4 - 7 . 464 - 0 / 



Equation ( 21 ) requires us to determine F , the smallest eigenvalue of V VT 1 . 

— s—d. 

It is found that F = 1 . 45017 ., With df^ = dfg = 648 this v al ue is signifi- 
cant at a level 0 ! < .01 . We conclude that the trait captured by vocabulary 

tests depends on speed conditions . 

The significance level can also be determined by means of a t table if 
we use a result obtained by Cacoullos (1965) and later independently by Kristof 
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(in press). This result states that t = Jv (Jf - 1 //f)/ 2 follows a t 
distribution with df = v when F follows an F distribution with df^ = 
df^ = v . Above F corresponds to t = 4.76 with df = 648. Considering 
that a one-sided t test is required we determine that Oi < .0005 • 

Case III: We find from the initial data that 



Quart ic . (26) was found to have two real and two complex roots. The minimal 
correlation r as required by (21) became .1859* We convert this into 



as a one-sided t with df = 647 . We see that it is identical to the value 
of t in case II. The difference in degrees of freedom is immaterial. Hence 
the two methods gave the same result as expected for the present example. 





and, after symmetrization 



( 



57-7751 65.7976 

65.7976 75.8201 



) 




and obtain t = 4-76 . This value is to be interpreted 




is* »*i*rrn {*•. Thy.m ciw; 
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